PFL-MoE: Personalized Federated Learning Based on Mixture of Experts

Federated learning (FL) is an emerging distributed machine paradigm that avoids data sharing among training nodes so as to protect privacy. Under the coordination of FL server, each client conducts model using its own computing resource and private set. The global can be created by aggregating results clients. To cope with highly non-IID distributions, personalized federated (PFL) has been proposed improve overall performance allowing learn a model. However, one major drawback loss generalization. achieve personalization while maintaining better generalization, in this paper, we propose new approach, named PFL-MoE, which mixes outputs via MoE architecture. PFL-MoE generic approach instantiated integrating existing PFL algorithms. Particularly, PFL-MF algorithm instance based on freeze-base algorithm. We further enhancing decision-making ability gating network variant PFL-MFE. demonstrate effectiveness LeNet-5 VGG-16 models Fashion-MNIST CIFAR-10 datasets partitions.